DOCUMENT RESUME 



ED 353 300 

AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



TM 019 364 

Page Ronald C; De Puga, Ignacio Suarez 
Development and Cross"Cultural Application of a 
Competency Assessment Questionnaire, 
nay 92 

qJo-'/T"" P"""ted at the Annual Meeting of the 

(7ti L r ^"^"^t^i^l Organizational Psychology 

yth Montreal. Quebec, Canada, May 2, 1992) 
Reports - Evaluative/Feasibility (142) - 
Speeches/Conference Papers (150) 

MF01/PC02 Plus Postage. 

^^Administrator Evaluation; Adults; Comparative 
Te xnp; '^Competence; Construct Validity; Cross 
Cultural Studies; Culture Fair Tests; Job sLus' 
Occupational Tests; ^Questionnaires; *Test 
VanSltr""' ^-liability; Test Use; Test 

Eva1uljion°'?"'';'°" ^Competency Based 

Evaluation. Competency Tests; Self Report Measures 



and use of the Be^'^^rL^ r' ^t^ri S:x"bd!)"^"tL^^^ ^^^^^o,.... 

^"Cn^^^^^ cross ultt^;^co^^r1r:^^^ 

managers; and thft nip ftabif^r^f 

considered. A competency is a^'^^L y n7 har^iur^tic^^l^^'^'^ 

individual chat is causally related to "L^ ""'^'^'^''c of an 

performance on the iob- ^0^0^ effective or superior 

Managers L capIfaLs^ !; congruent with construct validity. 

Managers in ^ ^^^^TT'' 

from capitalistic managers. Cross-cul turf ^ , "'^ profiles 

competency/personality^nstrLn s^a^b: ;p:r e?;^ ^• 
cultural relativism, cross-cul tnroi • ^PP^^^d if linguistic and 

iten, translation ha;e been sufficten f development, and 

included. (RLC) sufficiently addressed. Nine handouts are 



-•:.:::.::i!!::!.s:::::i„. : 



U.t. OCPARTKICNT Of EDUCATION 
Office Ol EdoceliooiJ RiMtrch end improvement 
eOUC^T.ONAL^Rc^SOURCES,NFORMAT,^ 

originating it 
C Minor Changes have been made to improve 
reproduction quality 

e Points ot view or Opinions '^^J,^"", 
ment do not necessarily represent oM.Ciai 
OERI position or policy 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



CO 
CO 

CO 



Development and Cross-Cultural Application of a 
Competency Assessment Questionnaire 



Ronald C. Page, Ph.D. 
and 

Ignacio Suarez De Puga 
Hay/McBer 



Presented as a part of the Symposium, Transporting Managerial Assessment Across Cultures . 
V.J. Arnold, Chair. Presented at the Seventh Annual Conference, Society for Industrial and 
Organizational Psychology, Montreal, Quebec, May 2, 1992. 



ERIC 



Correspondence regarding this paper may be directed to Ronald C. Page, Ph.D., Hay/McBer, 
121 So. 8th Street, Minneapolis, MN 55402, phone (612) 339-0555 or Ignacio Suarez De Puga, 
Hay/McBer, Plaza De La Lealtad, No. 3-5th Floor, 28014 Madrid Spain, phone 
011-34-1-522-2830. 



BEST COPY AViLME 

2 



Introduction 

For the past 25 year, Hay/McBer has been investigating the identification of managerial 
critical success factors that differentiate star performers from average performers. Following the 
research of David McClelland (1973, 1984, 1985), a co-founder of McBer, we have developed and 
applied a competency modeling process for articulating the competencies that drive exceptional 
job performance. This process has been applied in over 250 studies involving managers from 
twenty four countries. 

Following IVIcClel land's concepts, Hay/McBer had traditionally assessed managers through 
"operant" rather than "respondent" techniques. Operant techniques involve open-ended situations 
in which individuals must generate behaviors. A good example of this is the Behavior Event 
Interview , an interview process in which open-ended questions probe an individual's actions and 
motives in past situations. This process helps record the individual's "operant" behaviors in 
response to situations. In contrast, respondent measures, such as self-report questionnaires and 
multiple-choice tests, have not frequently been used by competency researchers. This is because 
of the belief that respondent instruments are subject to response biases and consequently will 
not identify true competence. 

Because of our recent research on competency behaviors, in the past year we have 
developed and cross-culturally researched several self-report, (respondent) instruments. Our 
presentation today will address one of these instruments, the Behavior Description Index (BDI). 

Specifically, our presentation will address the cross-cultural development and application 
of a self-report competency assessment questionnaire, in doing this, we will briefly address: 1) 
the definition of a competency, 2) the core (cross cultural) management competencies, 3) the 
development of culturally-transportable scales, 4) the cross-cultural comparison of results, and 
5) the transportability of competencies as constructs. 

What is a Competency? 

A competency is an underlvinq characteristic of an individual which is causally related to 
effective or superior performance on the job (Boyatzis, 1982). They are the "critical success 
factors" that are demonstrated by superior performers. Competencies are what superior 
performers 1) possess as underlying characteristics, 2) demonstrate in more situations, and 3) 
apply with better results. 

Competencies are diverse and can be any human attribute that can be measured reliably 
and that can be shown to differentiate significantly between superior and average performr -s. 
As shown in Attachment 2, (Different Types of Competencies), they may consist of such things 
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as 1) Knowledge, 2) Skills, 3) Social Roles, 4) Self Image, 5) Traits, and 6) Motives. They are 
the underlying characteristics that when employed in a situation, predict skilled behavior, which 
in turn predicts job performance (Spencer & Spencer, in press). 

As shown in Attachment 3, competencies differ in the extent to which they can be taught. 
Knowledge and skills can be most easily taught and motives and traits are most difficult to teach. 
From a cost effectiveness standpoint, the rule is to hire for deep-seated characteristics and to 
develop knowledge and skills. However, most organizations do the reverse; they hire on the 
basis of educational credentials and assume that candidates come with, or can be indoctrinated 
with, the appropriate motives and traits. 

A final point that we would like to make is that competencies are what employers (i.e., 
customers who hire workers) tend to be looking for. In contrast, educators and developers (i.e., 
suppliers who "mold" workers) tend to deal with knowledge and skills. Like the manufacturer who 
doesn't attend to and provide what the customer wants, educators and psychologists may 
potentially be accused of focusing on the wrong things. The current "crisis in education" as 
evidenced by the SCANS report, is oriented toward redirecting our educational systems toward 
educating for competence rather than for skills (SCANS, 1991). 

The Core Management Competencies 

We recently undertook an analysis of over 200 competency studies to identify the Core 
Competencies that appear to be consistently related to job effectiveness. We content analyzed 
well over 1000 behavioral indicators (i.e., critical incidents), identified in interviews with managers 
from 24 countries, that drove successful job outcomes. By assuring the broad cultural diversity 
of the key differentiating behaviors that we analyzed, we helped assure the cross-cultural 
relevance of these competencies. 

Nineteen Core Competencies most often found to predict success in professional and 
managerial jobs were identified (Spencer & Spencer, in press). These are identified in the 
Handout and are grouped in the following five clusters: 

♦ Self Management - four competencies that relate to personal style and are 
generally associated with personal maturity. These relate to superior performers 
In a wide variety of jobs. 

♦ Cognitive - three competencies that relate to problem solving and cognitive style. 
They are important to many technical/professional and basic managerial jobs as 
well as jobs in strategic thinking and planning. 
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♦ Achievement - three competencies relating to taking individual initiative and 
personal responsibility for task accomplishment to a standard of excellence. They 
relate to entrepreneurial, sales, and knowledge-worker jobs where drive, 
persistence, and self-directed goal setting are important. 

♦ Influence - five competencies relating to influencing and interacting with others. 
They are important for managerial, sales, and staff jobs. 

♦ Managerial - four competencies relating to supervising and directing others. They 
are important for supervisory, leadership, political and educational jobs. 

The Development of Culturally Transportable Scales 

We developed the BDI after we analyzed eight personality instruments and found only a 
handful of items that appeared to assess our core competencies. We then decided to develop 
our own instrument and constructed items by stating the collected behavioral indicators in simple 
terms. Consultants from the U.S., U.K., Singapore and Spain helped in building this instrument 
(Page & Spencer, 1991). In developing this iristrument, we found the following issues to be 
important: 

♦ Linguistic Relativism : Whorf (1956) addresses a critically important issue in 
defining the concept of linguistic relativism. In essence, he says that people's 
perception of reality and behavior are defined by the structure and content of 
language. We found that certain words or concepts didn't translate into Chinese, 
Spanish or even the English dialect as used outside of the U.S. Consequently, we 
only included items that would linguistically convey the same meaning across 
cultures. Also, the choice of words in English was at times based on the ease of 
translating it into other languages. For example, we found the following idioms to 
not work even among English speaking cultures: "mad," "I figure that," and "mind 
boggier questions". We found that "angry," "I assume that," and "mind teaser 
questions" to be more transportable and cross culturally meaningful. 

♦ Cultural Relativism : An example of this can be drawn from our efforts to scale the 
instrument. We had anticipated a true-false or three-point Likert response scaie. 
Our consultants in Singapore strongly argued for a five point scale to 
accommodate the Asian perspective which sees things more in terms of shades 
of gray and where to say "no" or "false" is not polite. We found that two or three 
response alternatives would not permit sufficient room for obtaining meaningful 
behavioral description among Asians. The five - point Likert scale format appears 



to resolve this issue. 

♦ Multiple Translation : The items were developed and reviewed by consultants from 
several cultures before they were finalized. After developing the BDI, it was 
translated into Spanish, Dutch and Hungarian. We had our translators translate 
it and then had our consultants, who were fluent in English and their native tongue, 
translate it back to English. Certain consultants completed the instrument in both 
English and their native language and we compared the results. We found that 
the key to effective translation was that: 1) we translated for meaning, not words; 
2) items were behaviorally stated in terms of concrete rather than abstract 
behaviors; and 3) the instrument had simple declarative statements - at the sixth 
grade reading difficulty level. 

Cross-Cultural Comparisons of Results 

The BDI resulting from the above process consists of 218 items. An example item from 
each competency is presented in the Handout. Items were crafted in a content-balanced manner 
for all the scales, meaning that half the items on a scale describe high competence and half 
describe low competence. This was done to reduce the effects of social desirability and response 
sets. The competency scales consist of 10 to 12 items, were rationally keyed from our a priori 
definitions, and have a median alpha reliability of .71 with an international sample. 

There are two validity scales, Frankness and Behavioral Consistency. Frankness is like 
the K scale on the MMPI, and is designed to identify those who try to "fake positive". Behavioral 
Consistency consists of pairs of behaviorally opposite terms and is designed to detect those who 
are inattentive or inconsistent in responding. 

Figure 5 presents mean profiles from four diverse groups (representing all group data collected 
in the three months since introduction of this form): 

♦ A group of managers/consultants from Madrid, Spain (N = 12). They were 
described as aggressive, successful business people. 

♦ A group of managers from Budapest, Hungary (N = 15). This group has spent 
their working careers in a socialist economy and are trying to determine how to 
transform themselves into budding capitalists. 

♦ A group of managers/consultants from Jakarta, Indonesia (N = 15). These are 
successful business people who have been influenced by the Chinese 
entrepreneurial style. 
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♦ A group of secretaries from a large, New York Financial Services Company. They 
have been included as a contrast group, because most competency scales were 
designed to identify successful managers. 

The scale scores are percentalized raw scores with a median score of 55 and a median 
standard deviation of 13. The exception is Behavioral Consistency, with a mean of 78. 

This figure reflects significant group differences on ail competencies other than Self 
Control, Flexibility, and Behavioral Consistency (Univariate F< .01; t tests on Table 1). 

In general, the successful managers/consultants from Spain and Indonesia look very 
similar. Likewise, the Hungarian managers generally have a similar profile to that of secretaries. 
There are two exceptions to this. First, the secretaries are higher on Organizational Commitment. 
This scale was designed to assess organizational conformity, as is important for secretaries, and 
is reflective of corporate success rather than entrepeneural success. Secondly secretaries were 
generally high on Concern for Order. This is congruent with our previous research indicating a 
high concern for precr '*on and orderliness for secretaries. Table 1 presents scale reliability, mean 
and standard deviation data. 

The "capitalist" managers/consultants were generally highest on: Self Confidence, Analytic 
Thinking, Conceptual Thinking, Achievement Orientation, Impact and Influence, Organizational 
Awareness, Relationship Building, and all four of the management competencies. 

Transportabilitv of Competencies as Constructs 

Generally speaking, the competency scales on this instrument appear lo have effectively 
transported across cultures. Additional data (on an earlier version of the BDI) from a socialized 
mining industry in Belgium (which the government is closing down) reveals a profile similar to that 
from Hungary. Additional managerial profiles from the U.S. reflect profiles similar to that of Spain 
and Indonesia. 

Internal consistency reliabilities for the separate culture groups were very similar. In our 
analysis thus far, we have diagnosed three cultural differences that were not fully anticipated in 
our development of the instrument: 

♦ Directiveness : In general, American managers are willing to be more directive 
than those of other countries. They may be more confrontive and more likely to 
discipline employees. For example, European managers can't fire someone 
without taking on substantial governmental regulation and financial burdens. 
Preliminary data suggest that Americans may be more directive. 
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♦ Impact and Influence : The desire to influence otiiers tiirough complex strategies 
appears to be perceived as more Machiavellian in other cultures than it does in the 
U.S. Consistency on this scale (alpha) has been less for non-U,S. samples, 

♦ Frankness : This scale was designed around the admission of common foibles 
such as envy, honesty, pride, etc. Because of cultural relativism, this scale 
appears to be reflecting cultural differences. The more "macho" or Power Distance 
(Hofetede, 1980) business cultures of Spain and Indonesia reflect the need for 
great care in crafting items dealing with human foibles. Frankness, the reverse of 
faking positive, has probably been the most difficult construct to cross-culturally 
transport. 



Conclusion 

Our cross-cultural research has helped us identify the following key conclusions: 

♦ The core competencies that drive successful managerial performance appear to 
be relevant across cultures, 

♦ Respondent instruments, as represented by the BDI, may provide self-assessment 
data that is reliable, culturally transportable, and congruent with construct 
(nomological network) validity hypothesis. 

♦ Managers in capitalistic economies tend to have similar competency profiles. 

♦ Managers from socialistic economies tend to have different competency profiles 
from that of capitalistic managers, 

♦ Cross-cultural, self-assessment competency/personality instruments may be 
applied if sufficient attention has been directed toward: 

Linguistic and Cultural Relativism 
Cross-cultural item and scale development 
Item translation 

♦ The "global economy" is providing an ever increasing need for psychologists to 
identify the pan-cultural, critical success factors of managerial effectiveness. 

We look forward to further advancements in this important area 
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Behavioral Description Index — Definition of Scales 



Self-Management 
Cluster 



Cognitive 
Cluster 



1 . Self-Control — ^The ability to keep one's emotions under control when provoked, when 
faced with opposition or hostility from others, or when working under conditions of 
stress. This includes the ability to restrain negative emotion or action in oneself. 

+ I stay calm and objective in situations of stress or conflict 

2. Self-Confidence — A belief in one's own capability to accomplish a task and select an 
effective approach to a task or problem. This includes confidence in one's own decisions 
or opinions, and the abilit>' to assert one's perspective or opinion. 

— I don't like talking in front of a group of people. 



3. 



4. 



5. 



6. 



7. 



Organizational Commitment — The ability and willingness to align one *s own behavior 
with the needs, priorities and goals of the organization. This includes respecting 
organizational rules, matching one*s behavior to the organization's expectations, 
effectively handling confidential matters, and aligning one's priorities with the 



organization s mission. 



+ I carefully match my behavior to the organization's expectations. 

Flexibility — The ability to adapt to and work effectively with a variety of situations, 
individuals or groups. This includes the ability to adapt to changes in one's own 
organization or job requirements, the ability to understand and appreciate different and 
opposing perspectives on an issue, and to adapt one's approach as the requirements of 
a situation change. 

— I don't like changing the way I do things. 

Analytical Thinking — The ability to break or analyze a complex problem, process or 
project into its component parts and to consider the parts in a systematic way. This 
includes the ability to make systematic comparisons of different features or aspects; or 
setting priorities, time sequence, and causal if*^ then relationships. 

+ I am good at anticipating problems and planning ways to deal with them. 

Conceptual Thinking — The ability to identify patterns or connections between 
situations that are not obviously related to identify the key or underlying issues in 
complex situations. This includes creative, conceptual or inductive reasoning. 

— I have difficulty seeing connections or patterns in complex situations. 

Information Seeking — Collecting and using relevant information on work-related 
problems or issues. This includes digging deeper, getting several opinions orinputs, and 
carefully investigating issues and known facts before making decisions. 
+ I check several sources of information before making up my mind. 
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Concern for Order — A concern for minimizing errors and maintaining high standards 
for quality of work. This is expressed in such forms as monitoring and checking work 
or information, keeping orderly records, and carefully planning tasks, 

— I don't bother keeping detailed records, 

9. Initiative — The tendency to act on one's own, and be persistent, to go beyond the job 
description, beyond what is asked, in order to get the job done. This includes being self- 
directed, seizing opportunities, and undertaking extraordinary personal efforts. 

— I have a tendency to give v.p when I meet difficult problems, 

10. Achievement Orientation — The desire to achieve, improve performance, and do a task 
better by committing oneself to accomplishing challenging objectives, or competing 
against a self-defined standard of excellence. This includes a willingness to commit 
significant effort in uncertain situations to reach challenging goals, 

+ I like to set challenging goals for myself and then measure my performance against 
related standards. 

Influence 1 1 . interpersonal Understanding — ^The ability to understand and interpret other individuals' 

Cluster concerns, .notives and feelings, and a tendency to take the time to try to understand 

others, 

— I have difficulty understanding the unspoken concerns or feelings of others, 

12. Customer-Service Orientation — A desire to help or service others. This includes 
focusing one's efforts on discovering and meet jig the customer's or client's needs. 

+ I like to work together with customers in solving their problems. 

13. Impact and Influence — The desire to impact others and the ability to influence or affect 
others through persuasiveness and influence strategies. This includes taking calculated 
steps to have a particular impact on others, 

+ I try to influence others by anticipating their reactions and preparing relevant 
arguments, 

14. Organizational Awareness — The ability to understand and learn the power relation- 
ships on one's own organization or in other organizations (customers, suppliers, etc). 
This includes the ability to identify who are the real decision makers and the individuals 
who can influence them; and to predict how new events or situations will affect 
individuals and groups within the organization, 

— I don't spend time trying to understand political relationships in the organization, 

15. Relationship Building — The tendency to develop and maintain a network or contacts 
both inside and outside one's own organization, with people who may be able to supply 
information, assistance, or support for work-related goals, 

— I don't like to socialize too much with people I work with. 



Achievement g. 
Cluster 
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Managerial 
Cluster 



Other 
Cluster 



16. Directiveness — The tendency to use the power of one's position in an effective and 
appropriate way, to enforce rules, to confront others about performance problems, to set 
boundaries for others' behavior, and to tell others what they must do. 

+ I don't hesitate to confront those who fail to "do" or "behave" as I expect 

17. Teamwork and Cooperation — The disposition to work cooperatively with others, to be 
a part of a team, to work cooperatively rather than working alone or competitively. 

— I don' t spend much time involving lots of people in making group plans or decisions. 

18. Developing Others — The disposition to try to effectively teach, train, develop or 
otherwise improve the skills of others. This includes taking steps or seeking out 
opportunities to develop others. 

+ I often encourage people in order to improve their motivation or performance. 

19. Team Leadership — The tendency to assume a leadership or motivational role when in 
group settings. This includes communicating goals, managing meetings, and motivating 
others. 

— I am not good at communicating a goal or objective that motivates others. 

20. Frankness — The tendency to be frank and honest in describing one's characteristics. 
This includes a willingness to admit that one is not extremely virtuous and that one does 
have minor foibles. This is used as a validity scale for identifying the respondents that 
may be attempting to look unusually good on the questionnaire. Reasonably honest 
respondent" should have a scc;e above 30. 

+ I have never repeated any gossip or rumors. 

2 1 . Behavioral Consistency — A consistency m describing one's self on the questionnaire. 
For pairs of items that represent opposite behaviors, high scoring respondents have 
tended to select opposite responses. This is used as a validity scale to identify individuals 
who are inattentive in lesponding or are making random responses. Properly completed 
questionnaires should have a score above 50. 

+ When somettung blocks what I am doing, I don't stop until I have found a way around it 

— I have a tendency to give up when I meet difficult problems. 



Note: Items with a "+" are positively keyed and items with a "— " symbol are negatively keyed. 
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2. Self Confidence 



3. Organizational Commitment 



4. Flexibility 



.75 



.66 



.71 



63 



52 



10.1 
10.8 



47** 
42* 



8.3 
9.9 



54 



12.1 



51 



9.6 



52** 
57* 



53 



13.8 
11.8 



12.1 



5. Analytic Thinking 



.74 



64 



10.6 



53* 



10.8 



54* 



11.4 



6. Conceptual Thinking 



.78 



60 



12.4 



47** 



9.4 



Information Seeking 



.74 



56 



9.6 



48* 



10.5 



49** 



47** 



15.0 



13.6 



8. Concern for Order 



.80 



64 



12.1 



44* 



10.8 



62 



15.6 



9. Initiative 



.57 



63 



7.6 



59 



7.8 



56** 



11.2 



10. Achievement Orientation 



.71 



63 



10.1 



48** 



8.1 



51" 



11.9 



1 1 . Interpersonal Understanding 



.69 



65 



8.3 



57* 



8.5 



12. Customer Serv. Orientation 



.68 



66 



10.4 



56** 



6.5 



55* 



62 



13.9 



11.4 



13. Impact and Influence 



.60 



50 



10.7 



38** 



8.9 



14. Organizational Awareness 



.70 



55 



12.1 



45* 



8.8 



38** 



43** 



'1.7 



15.2 



1 5. Relationship Building 



.71 



68 



1 6. Directiveness 



.73 



13.5 



50** 



55 



12.8 



44* 



8.1 
10.3 



55* 



40** 



13.6 



14.3 



17. Teamwork and Cooperation 



.71 



68 



10.7 



5V 



10.1 



56** 



12.7 



18. Developing Others 



.69 



60 



8.0 



52** 



6.0 



50** 



12.1 



1 9. Team Leadership 



.76 



66 



10.3 



52** 



7.3 



55** 



14.6 



20. Frankness 



.75 



39 



16.6 



55* 



13.2 



21 . BehavioraJ Consistency 



.78 



80 



4.9 



82 



4.3 



* P> .05 for t contrast with Spain & Indonesia 
**P> .001 for t contrast with Spain & Indonesia. 



50*^ 
77 



17.4 
8.5 



ERIC 
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